Applications involving Deep-Learning Neural Networks (NNs) or neuromorphic computing such as image recognition, natural language processing and more generally various pattern-matching or classification tasks are quickly becoming as important as general-purpose computing. The essential computational element of the NN, or neuron, multiplies of a set of input signals by a set of weights and sums the products. Thus, the neuron performs a vector-matrix product, or multiply-accumulate (MAC) operation. A NN generally includes a large number of interconnected neurons, each of which performs a MAC operation. Thus, operation of a NN is computationally intensive.
Performance of a NN may be improved by improving the efficiency of the MAC operation. It would be desirable to store weights locally to reduce power and the frequency of DRAM accesses. It may also be desirable to perform the MAC operation digitally to aid in reducing noise and process variability. A binary neuron may be capable of meeting these goals. Thus, a binary-weighted XNORNet has been developed.
In a binary XNOR cell, the weights, w, are mathematically 1 and −1 but are represented digitally as 1 and 0. The signals, x, are likewise mathematically 1 and −1, but represented digitally by 1 and 0. The result of the multiplication operation pi=wixi is positive only when x and w are both 1, and when they are both mathematically −1 (both 0 in Boolean representation). This is just the logical negation of the exclusive-OR operation (XNOR). Thus, the product of individual weights and signals can be expressed as pi=XNOR(wi,xi). The complete MAC operation for a given neuron is expressed as
      sum    =                  ∑                  {                      i            =            1                    }                n            ⁢                          ⁢                        w          i                ⁢                  x          i                      ,or in Boolean terms, sum=2 Count(XNOR(w,x))−n. The count operation counts the number of non-zero results of the XNOR expression, while n is the total number of inputs to the neuron. The result is then thresholded against a bias, resulting in a high or low output of the neuron. The entire process is digital. Consequently, no information loss associated with analog processing is incurred.
However, the use of the binary representation of the weights can be a source of information loss. Binary networks typically use substantially more neurons than analog (or multi-bit digital) networks to obtain the same level of overall accuracy. A significant improvement may be possible if the weights are ternary, rather than binary. Ternary weights take on the mathematical values −1, 0, and 1. The 0 weight produces the −1 output (logical 0) for any combination of inputs. Thus, the output of a ternary XNOR gate (also referred to as a “Gated XNOR”) is given by:
      p    i    =      {                                        XNOR            ⁡                          (                                                w                  i                                ,                                  x                  i                                            )                                                                          if              ⁢                                                          ⁢                              w                i                                      =                                          -                1                            ⁢                                                          ⁢              or              ⁢                                                          ⁢              1                                                                        -            1                                                              if              ⁢                                                          ⁢                              w                i                                      =            0                              
When performing the XNOR operation in the above equation, the nonzero weights and all signals are mapped from the {−1, 1} domain to the {0, 1} Boolean domain. The mapping is performed after the branching based on the mathematical value of the weight.
Ternary networks may provide improved accuracy over binary networks when using the same number of neurons. Alternatively, ternary networks can achieve the same level of accuracy as binary networks, but with a smaller number of neurons. This results in savings on area, power, and inference throughput and latency. Thus, both binary and ternary digital XNOR networks may be used in applications such as NNs. What is desired is an improved XNOR logic cell to enhance operation of digital binary and/or ternary NNs or other logic operations.